Collaborative Spatio-temporal Feature Learning for Video Action Recognition

Spatio-temporal feature learning is of central importance for action recognition in videos. Existing deep neural network models either learn spatial and temporal features independently (C2D) or jointly with unconstrained parameters (C3D). In this paper, we propose a novel neural operation which encodes spatio-temporal features collaboratively by imposing a weight-sharing constraint on the learnable parameters. In particular, we perform 2D convolution along three orthogonal views of volumetric video data,which learns spatial appearance and temporal motion cues respectively. By sharing the convolution kernels of different views, spatial and temporal features are collaboratively learned and thus benefit from each other. The complementary features are subsequently fused by a weighted summation whose coefficients are learned end-to-end. Our approach achieves state-of-the-art performance on large-scale benchmarks and won the 1st place in the Moments in Time Challenge 2018. Moreover, based on the learned coefficients of different views, we are able to quantify the contributions of spatial and temporal features. This analysis sheds light on interpretability of the model and may also guide the future design of algorithm for video recognition.Comment: CVPR 201

A Layer Decomposition-Recomposition Framework for Neuron Pruning towards Accurate Lightweight Networks

Neuron pruning is an efficient method to compress the network into a slimmer one for reducing the computational cost and storage overhead. Most of state-of-the-art results are obtained in a layer-by-layer optimization mode. It discards the unimportant input neurons and uses the survived ones to reconstruct the output neurons approaching to the original ones in a layer-by-layer manner. However, an unnoticed problem arises that the information loss is accumulated as layer increases since the survived neurons still do not encode the entire information as before. A better alternative is to propagate the entire useful information to reconstruct the pruned layer instead of directly discarding the less important neurons. To this end, we propose a novel Layer Decomposition-Recomposition Framework (LDRF) for neuron pruning, by which each layer's output information is recovered in an embedding space and then propagated to reconstruct the following pruned layers with useful information preserved. We mainly conduct our experiments on ILSVRC-12 benchmark with VGG-16 and ResNet-50. What should be emphasized is that our results before end-to-end fine-tuning are significantly superior owing to the information-preserving property of our proposed framework.With end-to-end fine-tuning, we achieve state-of-the-art results of 5.13x and 3x speed-up with only 0.5% and 0.65% top-5 accuracy drop respectively, which outperform the existing neuron pruning methods.Comment: accepted by AAAI19 as ora

User-differentiated hierarchical key management for the bring-your-own-device environments

To ensure confidentiality, the sensitive electronic data held within a corporation is always carefully encrypted and stored in a manner so that it is inaccessible to those parties who are not involved. During this process, the specific manners of how to keep, distribute, use, and update keys which are used to encrypt the sensitive data become an important thing to be considered. Through use of hierarchical key management, a technique that provides access controls in multi-user systems where a portion of sensitive resources shall only be made available to authorized users or security ordinances, required information is distributed on a need-to-know basis. As a result of this hierarchical key management, time-bound hierarchical key management further adds time controls to the information access process. There is no existing hierarchical key management scheme or time-bound hierarchical key management scheme which is able to differentiate users with the same authority. When changes are required for any user, all other users who have the same access authorities will be similarly affected, and this deficiency then further deteriorates due to a recent trend which has been called Bring-Your-Own-Device. This thesis proposes the construction of a new time-bound hierarchical key management scheme called the User-Differentiated Two-Layer Encryption-Based Scheme (UDTLEBC), one which is designed to differentiate between users. With this differentiation, whenever any changes are required for one user during the processes of key management, no additional users will be affected during these changes and these changes can be done without interactions with the users. This new scheme is both proven to be secure as a time-bound hierarchical key management scheme and efficient for use in a BYOD environment

Low-mass Active Galactic Nuclei on the Fundamental Plane of Black Hole Activity

It is widely known that in active galactic nuclei (AGNs) and black hole X-ray binaries (BHXBs), there is a tight correlation among their radio luminosity ($L_R$), X-ray luminosity ($L_X$) and BH mass (\mbh), the so-called `fundamental plane' (FP) of BH activity. Yet the supporting data are very limited in the \mbh regime between stellar mass (i.e., BHXBs) and 10$^{6.5}$\,\msun\ (namely, the lower bound of supermassive BHs in common AGNs). In this work, we developed a new method to measure the 1.4 GHz flux directly from the images of the VLA FIRST survey, and apply it to the type-1 low-mass AGNs in the \cite{2012ApJ...755..167D} sample. As a result, we obtained 19 new low-mass AGNs for FP research with both \mbh\ estimates (\mbh \approx 10^{5.5-6.5}\,\msun), reliable X-ray measurements, and (candidate) radio detections, tripling the number of such candidate sources in the literature.Most (if not all) of the low-mass AGNs follow the standard radio/X-ray correlation and the universal FP relation fitted with the combined dataset of BHXBs and supermassive AGNs by \citet{2009ApJ...706..404G}; the consistency in the radio/X-ray correlation slope among those accretion systems supports the picture that the accretion and ejection (jet) processes are quite similar in all accretion systems of different \mbh. In view of the FP relation, we speculate that the radio loudness $\mathcal{R}$ (i.e., the luminosity ratio of the jet to the accretion disk) of AGNs depends not only on Eddington ratio, but probably also on \mbh.Comment: ApJ accepte